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REMARKS 

INTRODUCTION 

Claims 1-7 were previously pending. 

Claims 1-7 stand rejected. 

Claims 1 , 2, and 4-7 are amended herein. 

Claims 8-13 have been added herein. 

Claims 1-13 are now pending and under consideration. 

No new matter is being presented, and approval and entry are respectfully requested. 

OBJECTION TO DRAWINGS 

The drawings have been amended to illustrate coordinate systems and optical axes. 
Figures 5 and 6 have been labeled "Prior Art." 

The requirement to label Figures 1 1-13 as "Prior Art" is respectfully traversed. Figures 
11-13 show features discussed in the Detailed Description of the present invention. The present 
specification does not characterize these figures as prior art. The Examiner is respectfully 
requested to cite a prior art reference indicating that the subject matter in Figures 1 1-13 is prior 
art. It is also submitted Figures 11-13 are not prior art because they show techniques for 
generating teaching model data used later for pattern matching to determine a position or 
direction of a model. As discussed below, this is not found in the prior art. 

Withdrawal of the objection is respectfully requested. 

PRESENT INVENTION 

In one aspect, the present invention relates to using a robot to capture images of a 
reference, object, according to different directions, orientations, positions, arrangements, etc. 
Each image is associated with respective information indicating the arrangement etc. when the 
image was taken. The image data or images can be stored and then used to automatically 
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determine orientations of subjects or objects with shapes similar or identical to the reference 
object. 

As recited in new claim 12, an aspect of the present invention may be thought of as: 
robotically taking images of a subject with different subject-camera arrangements and 
associating with each image information indicating its subject-camera arrangement; then taking 
a current image of a workpiece shaped like the subject; and then determining a current 
workpiece-camera arrangement by matching one of the images with the with current image, and 
using both the known arrangement of the camera and the predetermined subject-camera 
arrangement of the matched image to determine the arrangement of the workpiece relative to 
the camera (see new claim 12). 

PRIOR ART 

Suzuki discusses a system for analyzing images of an object viewed from various angles 
to determine the construction of the object. For example, Suzuki determines the positions and 
directions of edges and components of the object. This de-construction in Suzuki is then used 
to determine a program of robot instructions for assembling the object. 

As discussed below with reference to the claims, Suzuki does not discuss determining a 
current position/direction of an object using pattern matching and the predetermined image and 
associated position/direction information. 

REJECTIONS UNDER 35 USC § 112, SECOND PARAGRAPH 

In the Office Action, at pages 3-4, claims 1-3 were rejected under 35 U.S.C. § 112, 
second paragraph, for the reasons set forth therein. The claims have been amended. 
Withdrawal of the rejection is respectfully requested. 

OBJECTED DRAWINGS 

The drawings have been corrected as suggested, with the exception of Figures 11-13. It 
is respectfully submitted that no prior art teaches the content of Figures 11-13. Furthermore, 
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such content is not prior art because of its use in conjunction with patentably distinct features 
discussed above. For example, lines 10 and 11 on page 11 of the present specification 
mentions "[t]he two-dimensional array data produced as discussed above [with reference to 
Figs. 11-13] may be used as the image data to generate teaching models". 

REJECTIONS UNDER 35 USC §§ 102 AND 103 

In the Office Action, at pages 4 and 5, claims 1 , 3 and 4 were rejected under 35 U.S.C. 
§ 102 as anticipated by Suzuki (USPN 4,835,450). In the Office Action, at pages 5-7, claims 2, 5 
and 6 were rejected under 35 U.S.C. § 103 as being unpatentable over Suzuki in view of Werth 
et al. (USPN 4,504,970). Claim 7 was rejected in further view of Ninomiya (USPN 4,61 1 ,292). 
These rejections are traversed and reconsideration is requested. 

Claim 1, for example, recites "a current three-dimensional position and/or posture of 
[said] the subject object is recognized by carrying out pattern matching processing of an image 
of the subject based on [of] a plurality of pre-determined teaching models of the reference 
object". In contrast, Suzuki uses captured images of an object for de-constructing the object. 

As shown in Figure 1 of Suzuki, Suzuki uses captured images to pre-detect the 
construction of an object. Information reflecting the detected construction is then used for task 
and motion planning, which then controls the motion of a robot when constructing a like object 
("Accordingly, when a product is constructed in the same manner as described in FIGS. 8A to 
8D, the task sequence is set on the basis of a predetermined constructing rule"). Suzuki 
mentions the robot using a camera during construction, but it does not mention any detail of how 
that camera is used. The only discussion of using the camera during construction is: "an 
operation of moving the part is performed by correcting an operation of robot 3 in accordance 
with feedback signals from the pair of CCD cameras 1 1 a and 1 1 b, and pressure sensors 8 and 
9". Suzuki's CCD camera could be used, for example, in a simple manner such as identifying 
the presence or location of a part. 

Suzuki's moving robot does not use a camera or image pickup device to capture another 
image used for pattern matching. Rather the robot moves according to the detected 
construction; the motion is pre-planned. Suzuki's claim 1 further supports this understanding. 
Suzuki's claim 1 discusses "measuring a 3-dimensional form ... by imaging ... from a plurality of 
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directions", "detecting the construction from the measured 3-dimensional form", and generating 
movement task sequences based thereon. Suzuki does not discuss using the images captured 
from different directions for any purpose other than detecting the construction of the object. 

In sum, Suzuki does not discuss using an image of a subject and pattern matching on 
previously captured images to determine a position or orientation of the object after such images 
have been captured. The other references were not cited for, and do not add or suggest the 
feature discussed above. 

The difference in how claim 1, for example, uses pre-captured images, and how Suzuki 
uses images, is further apparent in claim 1's recitation of a "reference object" and a "subject 
object". The "reference object" is imaged in advance. Later, a direction or position of a "subject 
object" is determined using pattern matching with the images of the "reference object". Suzuki 
does not discuss this feature, but rather discusses capturing images of only one object, and 
processing the images at one time to de-construct the object. Later use of a camera is only 
generally mentioned. 

Various of the claims have been amended to remove "means" language, and have not 
been narrowed by such amendments. 

Withdrawal of the rejection is respectfully requested. 

DEPENDENT CLAIMS 

The dependent claims are deemed patentable due at least to their dependence from 
allowable independent claims. These claims are also patentable due to their recitation of 
independently distinguishing features. For example, claim 4 recites "said teaching model is 
composed of data obtained by performing image processing on the image data of the reference 
object". This feature is not taught or suggested by the prior art. Withdrawal of the rejection of 
the dependent claims is respectfully requested. 

NEW CLAIMS 

New claims 8-13 have been added to clarify an aspect of the present invention in which a 
current image of a subject is pattern-matched against the previous to determine a current 
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arrangement, orientation, etc. of the subject with respect to the current arrangement etc. of the 
image capture device. 

CONCLUSION 

There being no further outstanding objections or rejections, it is submitted that the 
application is in condition for allowance. An early action to that effect is courteously solicited. 

Finally, if there are any formal matters remaining after this response, the Examiner is 
requested to telephone the undersigned to attend to these matters. 

If there are any additional fees associated with filing of this Amendment, please charge 
the same to our Deposit Account No. 19-3935. 



Respectfully submitted, 



STAAS & HALSEY LLP 





700 Eleventh Street, NW, Suite 500 
Washington, D.C. 20001 
(202)434-1500 
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VERSION WITH MARKINGS TO SHOW CHANGES MADE 

IN THE SPECIFICATION: 

Please AMEND the paragraph that begins at page 8, line 2 with the following: 

An image produced by the image pickup device 20 is stored in the image memory 26. 
The image processor 33 conducts processing of the image stored in the image memory 
according to a command from the processor 31 to recognize the object. The image processing 
device 30 is similar in configuration and function to an image processing device in the prior art. 
However, according to the present invention, the image processing device 30 is characterized in 
that teaching models (described later) are stored in the non-volatile memory 37, and pattern 
matching is carried out by using the stored teaching models, for the image data of the pile of 
workpieces photographed by the image pickup device 20 to obtain three-dimensional position 
and posture of the workpiece. 

IN THE CLAIMS: 

Please AMEND the claims in accordance with the following: 

1 . (ONCE AMENDED) A teaching model generating device for image processing, in 
which [an] a subject object [or an article having] has the same or substantially similar shape as 
that of [the object is selected as] a reference object, and] , the device comprising: 

an image processing system with which a current three-dimensional position and/or 
posture of [said] the subject object is recognized by carrying out pattern matching processing of 
an image of the subject based on [of] a plurality of pre-determined teaching models of the 
reference object: and [,which were generated and stored] 

an image-capture system, in advance of the recognizing, generating and storing the 
plurality of teaching models on the basis of [the] respective image data produced by taking [the 
image] images of said reference object from a plurality of directions, [with the image data 
including the object,] wherein one of the reference object and [the] an image pickup device 
[means is fixed in place, while the other] is fixed to a movable part of a robot or is grasped with a 
hand of the robot, and said robot is operated for positioning to a plurality of different image 
pickup positions and directions [where direction of the optical axis of said image pickup means 
with respect to the reference object is different from one another respectively], so that the image 
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data respectively obtained at each of said different image pickup positions is stored as a 
teaching model. 

2. (ONCE AMENDED) A teaching model generating device for image processing, in 
which [an] a subject object [or an article having] has the same or substantially similar shape as 
that of [the object is selected as] a reference object, and] , the device comprising: 

an image processing system with which a current three-dimensional position and/or 
posture of [said] the subject object is recognized by carrying out pattern matching processing of 
an image of the subject based on [of] a plurality of pre-determined teaching models of the 
reference object; and [.which were generated and stored] 

an image-capture system, in advance of the recognizing, generating and storing the 
plurality of teaching models on the basis of [the] respective image data produced by taking [the 
image] images of said reference object from a plurality of directions, [with the image data 
including the object,] wherein the reference object is fixed to a movable part of a first robot pr is 
grasped with a hand of the first robot, and an image pickup means is fixed to a movable part of a 
second robot or is gasped with a hand of the second robot, and any one of or both of said first 
and second robots is operated for positioning to a plurality of different relative image pickup 
positions and directions [where direction of the optical axis of said image pickup means with 
respect to the reference object is different from one another respectively], so that the image data 
respectively obtained at each of said different image pickup positions is stored as a teaching 
model. 

3. (AS ONCE AMENDED) A teaching model generating device according to claim 1 , 
wherein said teaching model is a part of the image data of the reference object. 

4. (TWICE AMENDED) A teaching model generating device according to claim 1, 
wherein said teaching model is composed of data obtained by performing [a] image processing 
on the image data of the reference object. 

5. (TWICE AMENDED) A teaching model generating device according to claim 2, 
wherein said teaching model is generated for every direction in which said image pickup [means] 
device took the image of said reference object and said teaching model is stored in association 
with [the] information on the direction. 
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6. (TWICE AMENDED) A teaching model generating device according to claim 2, 
wherein said image pickup [means] device is a camera. 

7. (TWICE AMENDED) A teaching model generating device according to claim 2, 
wherein said image pickup means is a three-dimensional visual sensor whose image pickup 
[means] device measures the distance between the image pickup [means] device and a plurality 
of points on the object. 

8. (NEW) A method for teaching model generation and image processing, 
comprising: 

determining a current three-dimensional position and/or posture of a subject object by 
carrying out pattern matching processing of an image of the subject based on a plurality of 
predetermined teaching models of a reference object; and 

in advance of the determining, generating and storing the plurality of teaching models on 
the basis of respective image data produced by taking images of said reference object from a 
plurality of directions, wherein one of the reference object and an image pickup device is fixed to 
a movable part of a robot or is grasped with a hand of the robot, and said robot is operated for 
positioning to a plurality of different image pickup positions and directions, so that the image 
data respectively obtained at each of said image pickup positions is stored as a teaching model. 

9. (NEW) A method of automatic orientation recognition, comprising: 
generating and storing a set of images of different relative orientations of a subject, the 

images having been captured by a plurality of robotic operations corresponding to the different 

relative orientations of the subject, and associating with each image information indicating its 

respective relative orientation of the subject; 

with a known current orientation of a robot, capturing a current image of a workpiece that 

has an unknown orientation relative to the robot, where the workpiece has a shape substantially 

similar to the shape of the subject; 

using pattern matching to match one of the stored images with the current image; and 
determining the orientation of the workpiece relative to the robot based on the relative 

orientation information associated with the matched stored image, and also based on the known 

current orientation of the robot. 
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10. (NEW) A method according to claim 9, further comprising automatically 
maneuvering the robot to the workpiece based at least on the determined orientation of the 
workpiece relative to the robot. 

11 . (NEW) A method according to claim 9, wherein the generating and storing is 
performed for a plurality of differently shaped subjects, wherein the current image includes a 
plurality of differently shaped workpieces, and wherein the pattern matching further comprises 
identifying the workpiece from among the plurality of differently shaped workpieces using the 
images and orientation information of the plurality of differently shaped subjects. 

1 2. (NEW) A method of automatic orientation recognition, comprising: 
generating and storing a set of images of different relative arrangements of a subject, the 

images having been captured by a plurality of robotic operations corresponding to the different 

relative arrangements, and associating with each image information indicating its respective 

relative arrangement of the subject; 

with a known current arrangement of a robot, capturing a current image of a workpiece 

with an unknown current arrangement relative to the robot, where the workpiece has a shape 

substantially similar to the shape of the subject; 

using pattern matching to match one of the stored images with the current image; and 
determining the current arrangement of the workpiece relative to the robot based on the 

relative arrangement information associated with the matched stored image, and also based on 

the known current arrangement of the robot. 

1 3. (NEW) A method, comprising: 

robotically taking images of a subject with different subject-camera arrangements, and 
■* associating with each image information indicating its subject-camera arrangement; then 
taking a current image of a workpiece shaped like the subject; and then 
determining a current workpiece-camera arrangement by matching one of the images 
with the with current image, and using predetermined subject-camera arrangement information 
of the matched image to determine the arrangement of the workpiece relative to the camera. 
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